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RECOMMENDATIONS  REGARDING  THE  USE  OF  TEXTURES 
FOR  CUING  SURFACE  SLANT  AND  SHAPE  IN  FLIGHT  SIMULATION 

INTRODUCnON 

It  is  generally  recognized  that  texture  mapping  (or  cell  texturing)  yields  several  benefits  for 
flight  filiation:  It  provides  the  visual  grist  for  optic  flow,  wMch  is  an  important  cue  to  direction  of 
travel;  to  altitude  change;  and  to  speed  (shape  J&om  motion).  It  also  provides  information  that  allows 
one  to  instantaneously  perceive  the  local  slant  and  shape  of  the  terrain  (shape  firom  texture). 

Recent  research  into  the  perception  of  shape  firom  tecture  can  provide  the  theoretical  basis  for 
some  recommendations  regarding  the  optimal  choice  of  textures  for  these  purposes.  That  research  is 
briefly  summarized  here  as  it  applies  to  flight  mulation  Also,  the  relative  effectiveness  of  actual  cell 
textures  used  in  flight  simulation  are  compared  in  a  series  of  demonstrations. 

One  can  effectively  derive  3D  information  firom  the  systematic  effects  of  foreshortening  and 
perspective  on  surface  texture.  A  perspective  view  ofa  field  is  a  familiar  example:  As  one  scans  firom 
the  foreground  towards  the  horizon,  the  image  size  of  rocks  or  bushes  diminishes  "with  distance,  and 
their  density  increases  with  distance.  Continuous  changes  in  texture  size  and  density  across  an  image, 
known  as  texture  gradients,  effectively  provide  3D  information  about  the  slant,  distance,  and  3D  shape 
of  the  viewed  surfece. 

Gibson  (1950a,  1950b)  proposed  that  the  perception  of  visual  space  is  redudble  to  the 
perception  of  visual  surfaces.  ^edficaUy,  to  qualities  such  as  the  slant,  distance,  shape-at-a-gtyen-slant 
and  size-at-a-gjven-distance  at  locations  across  a  visual  surface.  He  fiirther  suggested  that  these  3D 
qualities  are  directly  related  to  specific  stimulus  variables,  placing  particular  emphasis  on  texture 
density.  Hypothesizing  that  the  spatial  properties  of  phenomenal  surfaces  are  in  correspondence  with 
variables  of  retinal  stimulation,  Gibson  proposed  that: 

Distance  may  be  given  by  the  relative  density  of  texture,  the  finer  the  density  the 

greater  being  the  distance.  Slant  at  any  point  may  be  ^ven  by  the  rate  of  increase  of 

density  of  the  texture  at  that  point.  Perspective,  in  this  theory,  reduces  to  the 
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increasing  condensation  of  the  projected  elements  of  the  phytical  environment  -  a 
gradient  of  density  along  a  meridian  of  the  retina  (Gibson,  1950b,  emphasis  his). 

Those  proposals  were  mathematical  oversimplifications,  as  they  did  not  take  into  contideration 
the  problem  of  isolating  the  effects  of  perspective  fi'om  those  of  foreshortening  on  texture  denaty,  as 
will  be  discussed.  Nonetheless,  Gibson’s  ideas  about  the  representation  of  visual  surfaces  in  terms  of 
local  spatial  properties  have  been  influential.  For  example,  subsequ^  studies  of  texture  gradients 
generally  have  assumed  that  the  observer  derives  apparent  local  sui&ce  orientation  or  depth  fi'om  the 
textured  image,  dther  fi'om  texture  density  or  some  local  measure  of  texture  compression  or 
foreshortening. 

Three  Visual  Percepts:  Orientation,  Curvature,  and  Distance 

A  basic  visual  ability  when  viewing  a  textured  surfiice  is  judging  the  orientation  of  the  surfece, 
which  entails  estimating  the  2  deg  of  fireedom  of  the  ^atial  orientation,  measured  relative  to  the 
observer.  These  are  usually  described  as  the  slant  and  tilt  of  the  surfece  (Stevens  1979, 1983a,  1983b). 
Slant  relates  to  foreshortening  and  is  measured  by  the  angle  between  the  normal  to  the  surfiice  and  the 
line  of  sight.  A  sur&ce  patch  of  zero  slant  is  one  seen  perpendicularly  (straight  on,  where  the  sur&ce 
normal  aligns  with  the  line  of  sight)  and  one  of  90  deg  slant  is  completely  foreshortened  such  that  the 
line  of  sight  grazes  the  sur&ce.  The  direction  of  slant,  termed  the  tilt  of  the  surface,  varies  throu^  360 
deg.  This  is  usually  defined  to  correspond  to  the  cfirection  of  the  (fistance  gradient  in  the  vidnhy  of  the 
surface  patch  or  equivalently,  the  image  direction  in  which  the  surfiice  normal  points.  Most  theories  of 
3D  perception  fi'om  texture  presume  that  the  results  are  the  recovery  of  local  surface  orientation. 

Given  surfece  stimuli  with  sufficiently  rich  3D  cues,  both  the  direction  and  magnitude  of 
surface  orientation  can  be  judged  with  admirable  precision.  Standard  deviations  in  apparent  tilt  can  be 
as  little  as  a  few  degrees  -vi^en  viewing  monocular  stimuli  (Stevens  1983a),  and  the  slant  of  a  surface 
can  be  matched  to  within  a  few  degrees  (Stevens  1983b).  The  subjects’  ability  to  estimate  orientation 
depends  on  whether  the  stimulus  surffices  present  sufficient  information  to  support  the  perceptual  task  - 
-  most  textures  do. 
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Cutting  and  Millard  (1984)  examined  the  relative  contributions  of  difi^ent  types  of  texture 
gradient  (gradients  of  texture  den^ty  versus  ^e  or  foreshortening  —  see  below)  towards  different 
perceptual  judgments.  They  concluded  that  the  visual  system  directly  extracts  curvature  from  texture 
(see  discussion  in  Cutting  1984,  Stevens  1984).  There  are  two  strat^es  for  percdving  surfrce 
curvature:  from  image  measures  that  vary  directly  with  suifrce  curvature;  and  indirect^,  by  first 
deriving  local  surfrce  orientation  from  measures  that  vary  directly  with  slant.  Both  are  likely  to  be 
involved  in  human  viaon.  Li  everyday  judgments  of  the  planarity  of  an  extended  surfrce,  it  is  necessary 
to  integrate  slant  information  across  different  visual  directions.  A  familiar  example  is  efrect  of  visual 
direction,  since  perspective  projection  will  cause  slants  to  vary  relative  to  the  line  of  sight  (recall  that 
the  visual  slant  of  a  floor  varies  from  zero  at  your  feet  to  45  deg  at  a  distance  of  one  eye  hdgjit  away 
from  your  feet).  Human  vidon  demonstrably  incorporates  such  kno\^dedge  in  its  ability  to  judge 
planarity  and  curvature. 

Texture  gradients  also  produce  the  impression  of  distance,  '\^ch  is  often  characterized  as  the 
computation  of  a  range  map  —  a  pointwise  representation  of  distance  from  the  observer  to  the  surfrce 
as  a  fimction  of  visual  direction.  A  distinct  type  of  distance  information  is  depth  (i.e.,  differoices  in 
absolute  distance  ~  such  as  across  a  surface  or  between  two  points).  Depth  is  related  to  sur&ce  relief 
or  the  amplitude  of  a  surface  as  it  extends  towards  the  observer,  and  is  often  expected  to  be 
represented  in  a  pointwise  manner,  as  a  so-called  depth  map. 

Humans  have  some  ability  to  convert  slant  information  into  depth  information.  To  d^onstrate 
this,  Stevens  and  Brookes  (1987)  used  a  binocular  depth  probe  to  measure  subjects’  apparent  depth  in 
monocular  surface  stimuli.  The  pictorially  represented  surfrces  were  presented  to  one  eye  and  a 
binocular  probe  point  was  momentarily  superimposed.  A  2AFC  e)q)erimental  paradigm  revealed 
apparent  depth  at  points  aaoss  the  monocular  surfece,  even  for  stimuli  drawn  in  orthograpluc 
projection.  This  conversion  among  three  forms  of  information,  slant,  curvature,  and  depth  (and 
perhaps  a  fourth,  wMch  is  distance),  makes  it  difficult  to  support  strong  assertions  as  to  the  form  of  the 
end  percept  of  3D  texture  perception.  The  question  ofan  end  percept  is  perhaps  not  well  founded:  If 
surface  perception  involves  processes  that  map  to  an  association  area  (depending  on  the  particular 
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pattern  of  inputs),  then  the  association  area  might  connect  all  percq)tual  knoniedge  of  the  surfece.  No 
single  form  of  3D  information  need  be  primary.  Regardless,  it  is  valuable  to  think  in  terms  of  these 
various  forms  of  3D  information  \^dien  attempting  to  account  for  \^diich  qualities  of  image  texture 
eflFectively  carry  3D  information. 

Two  Effects  of  Projection  on  Texture:  Compression  and  Scaling 

Fundamental  to  any  3D  perception  from  texture  is  the  task  of  inferring  (or  estimating)  3D 
shape  on  the  basis  of  image  measurements.  There  are  two  consequences  of  the  perspective  projection 
that  affect  the  imaging  of  physical  surface  texture  into  image  texture.  The  visual  system  must  make 
mfta.«girements  (or,  in  Other  terminology,  register  or  be  selectively  senative  to)  of  these  effects.  The 
effects  are  foreshortening  (the  anisotropic  compresaon  of  tetture  in  the  direction  the  surfece  is  slanted 
away  from  the  observer)  and  scaling  (the  isotropic  diminution  of  texture  image  aze  as  a  function  of 
distance  from  the  observe-.  A  planar  surface  is  subject  to  both  effects:  Its  texture  gets  progressivety 
smaller  and  foreshortened  in  the  distance.  AH  perceptual  effects  in  3D  texture  perception  can  be  traced 
to  these  two  fectors:  forKhortening  and  scaling.  Ifthe  visual  ^stem  can  make  measurements  that  are 
related  to  these  effects,  then  it  can  infer,  or  estimate,  slant  and  distance. 

The  sorts  of  measurements  that  have  been  proposed  include  texture  dement  density,  texture 
element  aze  or  area,  texture  element  width.  All  of  these  measurements  presume  discrete,  countable 
texture  elements  with  measurable  image  area  and  (hmensions  (see  below).  Also  proposed  are  more 
continuous  measurements  such  as  ^atial  frequency  content.  Additionally,  some  proposals  amount  to 
measuring  the  distortion  to  the  alhouette  shape  ofindividual  texture  elements.  CJenerally,  the  3D 
interpretation  of  such  measurements  requires  an  assumption  that  the  corresponding  physical 
dimensions  and  properties  are  constant  (or  regular,  or  isotropic,  etc.)  on  the  surface,  so  that  the 
measurements  reflect  the  consequences  of  the  projection  process  and  not  some  independent,  and 
misleading,  variation.  For  instance,  if  texture  size  is  presumed  constant,  then  variations  in  aze  can  be 
related  to  distance  (see  below). 
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The  3D  p^ception  of  sur&ce  slant  and  distance  from  image  texture  must  ultimately  be  based 
on  some  a  priori  assumptions,  because  3D  information  is  lost  in  the  process  of  projecting  to  a  2D 
image.  A  photograph  \Ndiichdiows  a  slanted  textured  surfrce,  is  of  course  produdng  an  illusion.  It  is 
difQcult  to  suppress  the  3D  interpretation  of  texture  variations  in  order  to  see  the  markings  on  the 
emulsion  of  the  photograph  veridically.  At  best  we  see  the  surfece  of  the  glos^  photograph  correctly, 
and  while  aware  that  the  maildngs  within  the  photograph  lie  on  that  emulsion  surfrce,  th^r  also 
strongly  represent  some  other  3D  surfrce.  This  point  is  crudal  to  the  dedgner  of  synthetic  textures. 
Textures  are  compelling  inducers  of  three-dimensionality.  Not  all  surface  textures  should  be  e?q)ected 
to  be  equally  effective  in  indudng  these  percepts,  however.  It  is  therefore  important  to  determine 
vdiich  texture  qualities  are  most  salient. 

Distance  from  Texture  Density 

Perspective  projection  affects  texture  denaty,  as  Gibson  (1950  a,  b)  observed.  Texture  denaty 
increases  towards  the  horizon  because  each  patch  of  unit  surfece  area  is  projected  in  the  image  with 
decreaang  scale  according  to  its  distance,  and  with  increasing  compression  with  its  fi)reshortening. 

This  also  increases  with  distance  for  a  plane  extending  towards  the  horizon. 

While  both  effects  act  in  the  same  direction,  the  mathematical  relationship  between  distance 
and  image  density  is  tractable  for  the  restricted  case  of  viewing  a  horizontal  planar  surface  (Bajcsy  & 
Liebermann,  1976;  Cutting  &  Millard,  1984;  Purdy,  1960;  Roanski,  1974).  Jau  and  Chin  (1990),  for 
instance,  show  an  approximate  solution  for  surface  oriartation  by  fitting  a  plane  to  measured  variations 
in  texture  density.  This  assumes  that  all  the  variation  is  due  to  perspective;  that  the  surface  is  a  plane; 
and,  as  usual,  that  the  texture  den^  is  uniform  across  the  surfrce.  Aloimonos  (1988)  shows  a  more 
general  solution,  but  one  that  again  assumes  the  surface  is  planar. 

Distance  from  Texture  Size 

In  contrast  to  the  problems  presented  in  interpreting  texture  density,  projected  size  provides,  in 
principle,  a  straightforward  cue  to  distance.  In  this  case,  the  validty  of  using  apparent  ^  to  infer 
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distance  rests  on  the  assumed  unifonmty  of  the  corresponding  phyacal  azes.  Thus,  for  example,  if 
across  a  dry  river  bed  the  size  of  stones  is  graded  from  large  in  one  re^on  to  small  in  another,  that 
systematic  variation  in  q>parent  aze  may  be  misinterpreted  from  certain  vantage  points  as  bdng  a  result 
of  perqrective.  Accepting  the  necessity  to  assume  the  physical  texture  aze  is  uniform,  how  might 
image  texture  size  be  measured? 

-^parent  aze  is  usually  attributed  to  individual  objects.  The  apparent  aze  (width  and  height) 
of  a  discrete  texture  element,  such  as  a  rock,  is  measured  as  a  subtaided  angle.  The  hdght  is  subject  to 
foreshortening,  such  as  a  flat  pebble  viewed  obliquefy.  The  width,  on  the  other  hand,  corresponds  to 
the  angular  measurement  which  is  least  likely  to  be  foreshortened.  Hence  it  wffl  vary  directly  with 
actual  width  and  inversely  with  distance.  The  width  measurement  is  usually  made  horizontalty,  as  in 
the  famihar  view  of  objects  on  a  slanted  horizontal  plane  extending  in  the  ^stance.  In  general, 
howevCT,  to  determine  the  width  (flmension  requires  analysis,  dther  of  the  apparent  direction  in  which 
elements  are  foreshortened,  or  of  the  apparent  direction  of  the  depth  gradient. 

In  the  case  of  viewing  a  smooth  slanted  surfrce  patch,  the  path  across  the  surface  along  which 
distance,  relative  to  the  observer,  changes  most  rapidly  corresponds  to  the  direction  of  the  distance 
gradient.  The  magnitude  of  the  (Ustance  gradient  is  trigonometrically  related  to  the  slant  angle;  the 
angle  between  the  surface  normal  and  the  view  vector.  For  an  extended  surfece,  a  field  of  gradient 
vectors  is  defined,  each  indicating  locally  the  direction  and  angle  in  which  the  surface  slants  away  from 
the  observer.  The  direction  of  the  distance  gradient  can  readily  be  determined  from  the  texture 
gradient  since  the  two  gradients  are  aligned.  Distance  varies  most  rapidly  in  the  direction  that  elemait 
size,  foreshortening  or  density  also  varies  most  rapidly.  Perpendicular  to  the  direction  of  the  gradient 
of  any  of  these  quantities  is  a  direction  which  corresponds  to  a  path  on  the  surfrce  which  is  parallel  to 
the  image  plane.  That  is,  image  measurements  taken  in  that  direction  correspond  to  phyacal  laigths 
that  are  parallel  to  the  image  plane  0.e.,  unforeshortened). 

Since  the  direction  of  the  gradient  is  orthogonal  to  the  direction  of  least  variation,  there  are  two 
ways,  in  principle,  to  establish  the  orientation  of  the  distance  gradi«it.  Distance  locally  varies  most 
rapidly  in  the  direction  of  the  texture  gradient  (e.g.,  measured  by  density)  and  perpendicular  to  the 
direction  of  least  texture  variation.  Of  the  two,  determining  the  local  image  direction  in  which  the 
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texture  is  most  uniform  provides  a  more  robust  and  accurate  estimate  of  the  gradient  direction, 
compared  to  measuring  the  direction  of  greatest  change  in  texture  (Stevens,  1981a). 

While  apparent  ^  or  width  measurements  are  usually  associated  individual  diso'ete 

elements,  there  have  been  proposals  to  measure  size  indirectly  (e.g.,  on  the  ba^  of  a  prominent  peak  in 
spatial  frequency  from  a  local  Fourier  analysis  [Bajcsy  &  Liebermann,  1976]).  Shifts  in  that  peak 
across  the  image  would  presumably  correspond  to  shifts  in  projected  texture  scale.  Jau  and  Chin 
(1990),  mentioned  earlier,  use  a  related  Wignar  distribution  for  measuring  spatial  frequency. 

Sui&ce  Orientation  from  Local  Gradients 

The  gradient  opautor,  or  grad,  is  mathematically  V  =  c!/5x  +  5/6y,  the  sum  of  two  first 
derivatives  of  a  continuous  function  takai  in  two  (arbitrary  but  orthogonal)  image  directions  x  and  y. 
The  grad  operator  has  been  proposed  to  be  a  useful  operation  to  perform  on  texture,  as  it  leads  to 
sinq)le  relationships  between  slant  and  some  texture  measure  such  as  denaty  or  size.  As  discussed,  the 
direction  of  the  gradient  corresponds  to  the  surfrce  tilt  t  the  direction  of  the  gradient  distance 

relative  to  the  observer).  Generally  ignoring  the  question  of  regularizing  the  image,  in  order  that 
density  or  size  rrught  be  efiectively  treated  as  a  continuous  variable,  slant  angle  s  is  related  to  the  local 
gradient  of  texture  denaty  r  by 

tan  a  =  p/3p 

where  the  density  gradient  is  normalized  by  the  magnitude  of  the  density  at  each  location  (Purrfy,  1960; 
Stevens,  1979).  Similar  expressions  relate  slant  and  other  texture  variables.  It  is  agnificant  that  these 
direct  computations  of  slant  do  not  reqinre  globally  uniform  texture. 

Surface  Orientation  from  Back  projection 

A  similarly  orientation-dependent  measure  is  texture  foreshorterung.  The  frmiliar,  but 
unrealistic,  illustration  of  foreshortening  involves  drcular  surface  markings  which,  on  a  slanted  surface, 
project  as  ellipses.  The  predictable  manner  in  which  a  drcle  compresses  into  an  ellipse  provides  the 
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baas  for  the  recovery  of  the  slant  (e.g.,  as  the  arc  cosine  of  the  ratio  of  projected  hdght  to  width 
[Flock,  1964]).  Note  that  the  width  and  height  measurements  of  each  projected  marking  would  be 
made  perpendicular  and  parallel  to  the  local  distance  gradient.  Hus  somewhat  contrived  example  of 
foreshortening  illustrates  a  more  general  problem  of  back  projection:  finding  that  surfece  slant  which 
maximally  accounts  for  the  projection  of  the  ori^nal  unforeshortened  shape  into  the  given  image  sh^e. 

Witkin  (1981)  showed  that  a  general  solution  to  the  back  projection  problem  for 
foreshortening  is  in  terms  of  the  ^stematic  effect  of  slant  angle  on  tangent  direction.  Slant  compresses 
texture  anisotropically,  biasing  the  distribution  of  contour  tangents  to  cluster  around  the  orientation 
perpendicular  to  the  (firection  of  the  depth  gradient.  The  tangents  to  the  surfece  markings  are 
preferentially  flattened  about  one  axis,  an  effect  that  is  illustrated  by  the  &miliar 
ellipse-as-slanted-circle.  And  yet  this  is  more  general,  in  that  it  applies  to  virtually  aiy  pattern  of 
texture  contours.  The  solution  to  the  back  projection  problem  is  to  find  that  slant  which  accounts 
maximally  for  the  variation  in  tangent  directions  in  the  image  texture.  This  method  assumes  that  the 
original  distribution  of  tangent  directions  is  unbiased  (i.e.,  isotropic,  as  are  the  tangents  around  a  circle). 
But  generality  comes  fi'om  the  &ct  that  the  tangent  measurements  need  not  arise  fi'om  the  boundaries 
of  connected  forms:  A  random  scattering  of  oriented  bars  (e.g.,  an  image  of  rice  kernels  or  straw 
scattered  on  the  floor,  results  in  a  biased  distribution  of  orientations  wHch  carries  sufficient  evidence 
for  foreshortening  to  estimate  the  slant  of  the  underiying  plane  [Vorhees  &  Poggio,  1989]).  Blake  and 
Marinos  (1989)  show  uniqueness  and  convergence  results  for  an  iterative  algorithm  that  solves  the 
texture  orientation  back  projection  problem. 

Two  recent  studies  bring  closure  to  the  theoretical  analysis  of  texture  foreshortening  (Carding, 
1991;  Malik  &  Rosenholtz,  1993).  Carding  has  shown  how  slant  and  tilt  can  be  derived  fi:om  local 
gradient  measures  such  as  prowded  by  the  Cabor  filter.  This  method  requires  texture  that  is  isotropic. 
On  the  other  hand,  Malik  and  Rosenholtz  show  how  to  derive  orientation  by  treating  the  image  texture 
locally  as  a  texture  distortion  due  to  perspective,  and  to  recover  orientation  fi-om  an  analysis  of  the 
changes  in  texture  across  the  image  as  an  affine  transform  between  image  patches.  In  their  analysis,  the 
texture  need  only  be  uniform,  not  necessarily  isometric.  These  two  analyses,  representative  of  sevwal 
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more  (e.g.,  Super  &  Bovik,  1991)  show  two  possible  computational  strate^es  for  the  interpretation  of 
texture. 


EMPIRICAL  OBSERVATIONS 

Slant  Underestimation 

Gibson  relied  largely  on  t&cture  gradient  studies  to  provide  empirical  evidence  for  theory  of 
direct  perception  (Gibson,  1950a,  1950b;  Purdy,  1960).  As  initially  stated,  there  should  be  a  direct 
psychophysical  correspondence  between  texture  variables  and  3D  percqpts.  But  that  hypothecs  was 
not  borne  orrt  by  the  psychophyacal  experiments  that  followed.  Apparent  slant  (usually  measured  by 
matching  the  visual  impresaon  of  slant  with  that  of  a  palm  board)  is  often  underestimated;  it  does  not 
vary  in  direct  correspondence  with  that  spedfied  by  the  torture  gradient  (see  review  by  Epstdn  &  Park, 
1964).  That  cast  doubt  on  the  strict  interpretation  of  Gibson’s  above  hypotheses,  prompting  his 
revised  notion  of  aflfordances  (Gibson,  1979). 

Underestimation  of  apparent  slant  is  now  a  &miliar  effect,  often  observed  in  viewing  a  stimulus 
that  presents  only  a  single  3D  cue,  such  as  a  texture  gradient.  This  ^ect,  regression  to  the  frontal 
plane,  varies  with  exposure  time  (Beck,  1960;  Freeman,1965;  Gibson,  1950b;  Purdy,  1960;  Smith  & 
Smith,  1957)  and  varies  condderably  with  the  quality  of  the  stimulus.  For  example,  much  less 
underestimation  is  observed  for  textures  that  contain  linear  p^pective  ^.e.,  conver^g  lines  in  the 
image  that  appear  parallel  in  3D  [Clarke,  Smith,  &  Rabe,  1956]). 

The  Efficacy  of  Texture  Dendty 

It  was  discussed  that  the  distribution  of  texture  density  across  an  image,  in  and  of  itself  does 
not  provide  adequate  constraint  on  the  3D  interpretation  of  the  underlying  surface,  unless  overly 
restrictive  assumptions  are  imposed.  In  prindple,  if  one  assumes  the  surfrice  is  planar,  or  furthermore 
horizontal,  one  can  solve  the  orientation  and  distance  parameters  from  the  image  texture.  But  since 
surfaces  are  not  so  constrained,  it  is  thus  not  surprising  that  terture  denaty  gradients,  in  isolation,  have 
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been  found  to  convey  little  eflfective  3D  information.  By  presenting  random  dot  patterns  as  stimuli, 
varying  local  dot  density  would  arguabfy  allow  isolation  of  the  denaty  cue  from  size,  foreshorteniiig, 
and  other  texture  gradient  cues.  Many  such  studies  have  shown  that  density  in  such  displays  is  an 
ineffective  ciie  to  3D  shape  and  distance  (Braunstein,  1966, 1968;  Braunstein  &  Payne,  1969;  Cutting 
&  Millard,  1984;  Smith  &  Smith,  1957;  Stevens,  1981a).  ffdot  density  increases  from  bottom  to  top 
according  to  a  slanted  plane  viewed  in  perspective,  it  is  possible  to  interpret  the  increasing  denaty  as 
suggesting  increasang  distance.  But  that  depth  impression  is  not  compelling;  The  same  display  mi^ 
suggea  curvature,  as  if  viewing  a  cylindo",  or  the  dots  may  also  simply  appear  to  lie  flat  in  the  image 
plane,  the  obvious  density  gradient  notwithstanding.  There  may  be  an  interaction  between  density  and 
other  3D  cues,  however,  such  as  a  weakening  of  the  impression  of  slant  when  uniform  image  density  is 
placed  in  conflict  with  the  slant  information  provided  by  other  texture  gradients  (Cutting  &  Millard, 
1984),  as  discussed  below. 


Texture  Size 

It  is  well  known  that  apparent  size  is  a  strong  cue  to  distance  in  accordance  with  perspective 
(Hochberg,  1971).  The  tendency  to  relate  apparent  size  and  apparent  distance  may  play  a  substantial 
contribution  to  the  impression  of  depth  in  texture  gradients,  but  not  be  restricted  to  the  case  of  features 
that  lie  across  a  common  surface.  Stevens  (1981a)  showed  that  elements  that  are  similar  exc^t  for 
size,  told  to  appear  distributed  in  depth  with  relative  distances  varying  inversely  by  their  aze, 
independent  of  their  pattern  of  distribution.  That  is,  size  gives  rise  to  distance  independent  of  whether 
the  elements  are  arranged  as  spatially  continuous  gradients  defining  a  continuous  plane,  or  a  random 
arrangement,  as  if  the  elements  were  scattered  throu^  a  volume  of  space.  Saisitivity  to  the 
size-distance  relationship  in  general,  therefore,  may  be  responsible  for  much  of  the  appearance  of  depth 
and  slant  spedfically  in  texture  gradient  images  wiiere  discrete  elements  present  a  continuous  rize 
gradient.  Obviously,  vdien  viewing  stimuli  as  simple  as  circles  of  different  image  diameters,  to  info" 
differences  in  distance  the  visual  system  must  assume  the  corresponding  3D  objects  have  equal 
diameter  but  different  distances  from  the  observer.  This  is  one  of  the  central  assumptions  underlying 
the  interpretation  of  texture  gradients,  to  which  we  now  turn. 
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UNDERLYING  ASSUMPTIONS  ABOUT  THE  PHYSICAL  TEXTURE 


Geometric  Requirements 

To  be  usefiil  sources  of  3D  information,  phyacal  texture  must  obey  several  restrictions; 
spatial  homogeneity.  To  be  a  valid  cue  to  distance,  the  phyacal  texture  statistics  must 
be  invariant  across  the  surtiice.  bi  particular,  a  gradual  variation  in  ^  would,  in  the 
absence  of  other  information,  erroneously  suggest  variation  in  distance. 

distinguishable  size.  To  be  a  valid  cue  to  distance,  a  suflBdently  narrow  range  of 
element  azes  must  be  present  in  any  vicinity,  so  that  projected  sizes  at  differoit 
localities  are  comparable. 

isotropy.  To  be  a  valid  cue  to  slant,  the  phyacal  texture  must,  according  to  most 
theories  of  human  perception,  be  isotropic,  at  least  not  systonatically  mirruc 
foreshortening.  In  particular,  the  surface  features  should  provide  contours  with 
random  correlation  between  tangent  direction  and  curvature. 

The  reqiurement  for  homogeneity  is  all  too  easily  met  in  the  amulator;  Memory  restrictions  for 
texture  mapping  usually  result  in  repetitious  textures,  often  to  the  detriment  of  the  overall  appearance. 
The  second  reqiurement,  distingtushable  size,  is  more  difficult.  Some  granular  textures  presort,  within 
a  given  ndghborhood,  at  least  an  order  of  magnitude  of  element  sizes.  If  the  distribution  of  azes  is 
broad  or  nearly  flat,  then  the  visual  tyaem  has  difficulty  distinguishing  a  statistical  diffierence  in  texture 
projected  size  across  the  image.  On  the  other  hand,  a  very  narrow  range  of  azes  would  appear 
artificial  and  improbable.  Although  the  effect  of  the  scatter  of  texture  size  on  the  accuracy  of  3D 
judgments  has  not  been  studied  by  formal  psychophyacal  experiments,  it  appears  that  the  visual  system 
tolerates  a  wide  distribution  of  sizes  provided  there  is  a  sharp  peak  in  the  distribution  which,  given 
spatial  homogendty,  is  constant  across  the  surface. 

Isotropy  is  required  to  derive  slant  from  foreshortening.  The  visual  system  is  sufficiently  robust 
to  often  ignore  the  erroneous  information  about  slant  from  anisotropic  textures.  For  example,  in  flight 
simulation,  texture  is  often  projected  or  mapped  downward  onto  the  slopes  of  the  terrain  (not  normally 
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to  the  polygons).  The  result  is  that  texture  is  stretched  in  the  direction  of  the  gradient  of  altitude,  and 
the  texture  on  steeply  sloping  terrain  looks  like  streaks.  Tlus  effect  is  widespread  in  texture  mapping, 
and  yields  an  incorrect  cue  to  slant.  Of  course,  if  the  texture  does  appear  streaked  or  stretched,  that 
correct  perc^tion  derives  from  otiiCT  cues  than  texture.  Of  concern  are  cases  where  the  orientation  of 
the  textured  surface  is  misperceived  based  on  the  erroneous  assumption  that  the  surfrce  is  isotropicalfy 
textured. 

In  addition  to  homogeneity  and  isotropy,  the  luminance  contrast  vith  the  texture  is  also 
important  in  order  to  extract  reliable  contour  foreshortening  information.  An  indeterminate  blotchy 
texture,  devoid  of  high  spatial  frequency  components,  is  very  ineffective  as  a  source  of  slant 
information. 

Highly  sali^t  cues  to  changes  in  sur&ce  tangent  are  linear  contours  that  lie  across  the  surfrce 
(Stevens,  1981b,  1983c).  Curvature  of  the  2D  contour  in  an  image  provides  information  about  the 
curvature  of  the  corresponding  3D  contour  across  the  surfece.  In  the  limit,  a  sharp  discontinuity  in 
tangent  in  the  2D  contour  is  a  local  cue  to  a  surface  CTease.  These  creases  often  arise  as  artifects  of 
texture  mapping  algorithms  (along  the  borders  of  the  cell  textures  ^^^e  they  abut  a  rqjeating  cell),  but 
despite  the  unrealistic  appearance,  these  linear  features  are  highly  salient  sources  of  information  about 
creases. 


Illustrations  Using  Texture  Mapped  Cell  Textures 

To  examine  the  relative  efficacy  of  actual  cell  textures,  six  photographs  of  actual  cell  textures 
were  pro\dded  in  order  to  gaierate  the  series  of  demonstrations  shown  below.  In  some  of  the  original 
photographs  the  wallpaper-like  tiling  of  the  cell  texture  was  apparent  (see  Figures  la-^  illustrating 
Textures  A-F).  The  ax  textures  were  scanned  and  di^tally  cropped  as  necessary.  Then  they  were 
texture  mapped  onto  a  variety  of  stimulus  surfaces  using  software  especially  written  for  this  task.  The 
goal  was  to  provide  a  series  of  still  photographs  of  representative  cell  textures  in  controlled  viewing 
drcumstances,  including  perspective  versus  orthographic  projection;  planar  versus  curved;  creases; 
occlusion;  and  different  magnifications.  Note  that  the  cell  texture  samples  have  undergone  a  long  series 
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of  image  processing  transformations:  The  emulator’s  cell  texture  ori^nated  as  a  photograph  (e.g.,  of 
an  acoustic  cdling  tile,  such  as  was  the  source  of  Texture  B),  which  was  then  scanned  in  at  the 
Armstrong  Laboratory  fecility  (at  Mesa,  AZ),  and  used  as  a  cell  texture.  For  this  study,  the  cell 
textures  were  first  photographed  on  a  computer  monitor  at  the  Armstrong  Laboratory  &dlity  and  then 
enlarged  and  printed.  At  the  University  of  Oregon  lab,  the  ^ossy  photos  were  scarmed  to  return  them 
to  digital  form  so  that  the  cell  textures  could  be  mapped  onto  a  distal  surface.  To  produce  the  final 
illustrations,  textures  were  projected  onto  a  computer  monitor,  photographed,  then  sent  to  the 
Armstrong  Laboratory  fecility  for  publication.  It  was  tiiCTefore  understandably  not  possible  to  preserve 
the  predse  grey  level  distributions  of  the  cell  textures;  nonetheless,  their  characteristic  geometries  are 
apparent  and  yield  different  results. 

Texture  A,  while  resembling  an  aerial  photograph  of  actual  terrain,  is  not  promidng  as  a 
texture  gradient  cue.  It  is  quite  inhomogeneous  and  composed  of  elements  of  a  broad  range  of  sizes, 
and  furthermore  is  anisotropic.  Texture  density  varies  consid^ably  across  the  figure  in  that  some 
regions  are  predominantly  light.  Others  are  predominantly  dark.  Moreover,  the  blotches  in  each 
locality  vary  over  a  very  large  range  of  scales,  making  it  difficult  to  correlate  aze  and  distance.  Finally, 
while  the  texture  blotches  are  inffividually  fairly  isotropic,  there  are  short  striations  in  different 
orientations.  In  contrast.  Texture  B,  ori^naUy  a  photograph  of  acoustic  ceiling  tile,  is  quite 
homogeneous,  presents  high-contrast,  has  convex  elements  of  distinguishable  aze,  and  is  quite 
isotropic.  Texture  C  is  moderately  homogeneous,  but  presents  a  large  range  of  element  scales  with 
indistinct  borders  and  often  low  contrast.  That  texture  is  fairly  isotropic.  Note  that  Texture  C  shows 
fovir  repetitions  of  the  basic  texture  tiled  together.  Texture  D,  showing  an  obvious  pattern  of  four-by- 
four  tiles,  is  a  rather  miserable  texture  with  inhomogeneous  density,  a  motley  collection  of  blotches  of 
indeterminate  size,  and  a  diagonal  bias  which  ftlsely  suggests  foreshortening,  even  in  this  unslanted 
view.  Texture  E,  a  four-by-four  tiling,  is  quite  homogeneous.  It  is  rather  distinct  in  size  and  is  slightly 
anisotropic.  Finally,  Texture  F  presents  large  irregular  viliite  blotches,  stark  contrast,  inhomogeneous 
density,  yet  isotropy.  How  do  these  six  textures  compare  as  sources  of  3D  information?  Textures  B 
and  £  should  be  effective  given  their  density  homogeneity  and  distinguishable  element  size.  Texture  F 
might  present  good  foreshortening  cues  based  on  its  isotropic  distribution  of  high-contrast  contour 
tangents. 
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A  2D  Gaussian  produces  a  smooth  hill  shape  for  our  purposes.  Viewed  in  perspective  the  hill 
is  apparent  not  only  because  of  the  texture  gradient  cue,  but  because  the  crest  of  the  hill  occludes  the 
more  distant  background  (Figure  2).  For  comparison,  Rgure  2  shows  Texture  A  mapped  over  this 
surfece.  The  scale  of  the  texture  relative  to  the  hill  was  aibitrarily  chosen  (scale  is  examined  below). 

Figure  3  shows  Texture  A  on  the  same  surface,  but  in  orthographic  projection  rather  than 
perspective.  The  primary  difference  is  the  removal  of  the  trapezoidal  outline  cue  to  3D  orientatioiL 

Figure  4  substitutes  Texture  B  for  A,  holchng  other  fectors  constant.  Note  how  the 
foreshortening  of  the  dark  blotches  varies  quite  distinctly  accorrhng  to  the  slope  of  the  hill,  yet  the 
occlusion  cue  along  the  CTest  of  the  hill  is  less  apparent. 

Despite  the  theoretical  expectation  that  Texture  B  (Figure  4)  should  be  more  effective  than 
Texture  A  (Figure  3),  that  actual  advantage,  if  any,  is  not  impressive.  On  the  contrary,  the  greater 
texture  density  and  the  fortuitous  position  of  the  dark  patch  in  Figure  3  makes  the  hill  stand  out  relative 
against  the  background.  The  robustness  ofhuman  vision  is  apparent  here:  It  is  difficult  to  fector  out 
the  contribution  of  shading  from  that  due  to  texture. 

To  factor  out  shading,  the  terrain  next  consists  of  two  planes,  one  horizontal,  one  sloped,  that 
meet  along  a  sharp  horizontal  crease  (a  smoothly  curved  join  would  be  6r  more  difficult  to  detect). 
This  approximates  an  important  visual  task  for  low  level  flight;  the  detection  of  differences  in  ground 
slope  ahead  of  the  aircraft.  Texture  differences  might  reveal  the  slope  change,  if  not  a  visually  distinct 
crease  itself  Each  of  the  six  textures  were  mapped  over  the  creased  surfece,  and  projected  so  that  the 
crease  is  horizontal  and  viewed  from  a  perspective  resembling  that  ahead  of  an  aircraft,  with  a  20  deg 
angle  of  depression.  The  crease  was  not  subtle,  consisting  of  an  up-slope  of  0. 12  or  6.8  deg.  The  sfac 
textures  are  shown  in  Figures  5a-f  While  the  crease  is  difficult  to  discern  in  all  cases,  in  Texture  C 
(Figure  5c)  the  unintentional  contrast  edge  across  the  border  of  the  tile  shows  a  discontinuity  in 
tangent,  a  cue  to  the  underlying  surface  crease.  Thus  the  linear  edges  associated  with  cell  texture 
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borders,  while  unrealistic,  may  be  useful  in  the  simulator.  Nonetheless,  some  scrutiny  is  still  required  to 
interpret  the  figure  in  3D. 

Figure  6  shows  the  sk  textures  in  orthographic  rather  than  perspective  projection.  Without  the 
perspective  splay  in  the  foreground,  the  figures  look  far  less  three-dimensional.  The  task  is  now  simply 
to  judge  how  discernible  is  the  discontinuity  in  texture  between  the  bottom  and  top  portions  of  the 
image.  Li  Figure  6b  (Texture  B)  the  foreshortening  is  noticeably  greater  below  the  crease  than  above, 
but  in  Figure  6c  (Texture  C)  the  vertical  edge  artifact  just  to  the  right  of  center  is  more  apparent  than 
the  horizontal  crease  (c£  Texture  C  in  perspective.  Figure  5c). 

The  detectability  of  the  crease  depends  on  the  scale  of  the  figure,  as  can  be  noted  by  observing 
them  at  different  viewing  distances.  Two  effects,  which  work  in  opposite  directions,  can  then  be  noted. 
Compression  gradients  are  more  apparent  as  the  texture  scale  is  reduced,  so  that  more  blotches  fall  in 
a  given  visual  area,  within  limits.  On  the  other  hand,  to  detect  foreshortening  or  compression,  the 
individual  blotches  must  be  discernible,  and  that  improves  with  increasing  scale.  Figure  7  shows 
different  scales  or  magnifications  (lx,  2x,  4x,  8x)  of  the  corresponding  displays  in  Figure  6;  as  before, 
the  horizontal  crease  is  approximately  40%  up  the  figure. 

Finally,  while  the  formal  psychophysics  are  not  in  the  literature,  it  would  appear  that  the 
detectability  of  differential  foreshortening  is  a  function  of  the  overall  slant.  Figures  6  and  7  correspond 
to  a  20  deg  angle  of  depression  or  a  slant  of  70  deg  (angle  between  the  line  of  sight  and  the  surface 
normal  for  the  lower  surface,  the  plane  below  the  crease).  Increasing  the  slant  to  80  deg  (Figure  8) 
makes  the  differential  foreshortening  much  more  apparent. 
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Figure  lb. 
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iithograpMc  versus  perspective  projes 
it,  thus  farther  isolating  the  contributi 


Figure  4. 

Same  as  Figure  3,  but  with  Texture  B  substituted  for  Texture  A. 


Figure  5a. 

A  creased  surface  is  viewed  in  perspective  for  textures  A-F. 
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Figure  6a. 

Same  as  Figure  5,  but  in  orthograpHc  projectior 


Figure  6b. 


Figure  6c. 
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CONCLUSIONS 


The  3D  informatioii  in  texture  is  carried  in  part  by  cues  associated  with  collections  of  texture 
elements,  such  as  the  texture  density,  and  in  part  by  the  properties  of  individual  elements,  such  as 
foreshortening.  The  former  cues  require  homogeneity  and  a  uniform,  distinguishable  size;  the  latter 
require  isotropy.  The  cell  texture  samples  we  examined  varied  in  quality  along  these  properties. 
Texture  B  appeared  particularly  high  quality  in  terms  of  homogeneity  and  rize  uniformity;  Texture  F 
provided  a  rich  and  isotropic  pattern  of  Mgh  contrast  contours.  The  corresponding  demonstrations 
qualtatively  reflect  these  advantages. 

In  short,  the  two  alternatives  exammed  are  to  have  large-scale  isotropic  blotches  with  moderate 
contrast,  or  to  have  small,  discrete,  homogeneous-sized  features.  The  prediction  would  be  that  an 
optimal  texture,  robust  over  a  large  range  of  scales,  would  be  one  that  combines  the  two:  one  with 
large-scale  blotches  that  viewed  normaly  to  the  surface,  appear  random  in  contour  tangent  direction; 
and  superimposed  on  this,  a  pattern  of  discernible  markmgs  resembling  those  in  Texture  B. 
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